Improved Guarantees for Agnostic Learning of Disjunctions

Given some arbitrary distribution D over {0, 1}n and arbitrary target function c∗, the problem of agnostic learning of disjunctions is to achieve an error rate comparable to the error OPTdisj of the best disjunction with respect to (D, c∗). Achieving error O(n · OPTdisj) + ǫ is trivial, and Winnow [13] achieves error O(r ·OPTdisj) + ǫ, where r is the number of relevant variables in the best disjunction. In recent work, Peleg [14] shows how to achieve a bound of Õ( √ n ·OPTdisj)+ ǫ in polynomial time. In this paper we improve on Peleg’s bound, giving a polynomial-time algorithm achieving a bound of O(n ·OPTdisj) + ǫ for any constant α > 0. The heart of the algorithm is a method for weak-learning when OPTdisj = O(1/n), which can then be fed into existing agnostic boosting procedures to achieve the desired guarantee.